Repository navigation
fix: read config through scalo and drop refused DLQ entries - #369
Merged
Merged
Conversation
--config still wins. Without it, scalo 2.14.33 reads the file DFE_LOADER_CONFIG names. scalo ignores a blank DFE_LOADER_CONFIG. The hot-reload watcher uses CommonArgs::config_path. The loader's resolve_config_path is gone. On DlqError::Refused, place_dead_letters drops the entries the error lists. The loader counts them in pipeline_dead_letters_dropped_total with reason too_large. The rest of the write counts as placed. When the broker refuses a dead letter, the batch can move on before the retry expires. Every other DLQ error keeps its retry. Both scalo minimum versions are 2.14.33 to pick up DlqError::Refused.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When --config is unset, scalo 2.14.33 reads the file DFE_LOADER_CONFIG names. It reports DLQ size refusals as DlqError::Refused.
Orchestrator unit tests check which entries are dropped and how many are counted. They use a predicate in place of RefusedEntries, which only scalo can build. A binary test checks DFE_LOADER_CONFIG. scalo tests refusal from a broker.